Joint Radiometric Calibration and Feature Tracking for an Adaptive Stereo System

To capture the full brightness range of natural scenes, cameras automatically adjust the exposure value which causes the brightness of scene points to change from frame to frame. Given such a video sequence, we introduce an adaptive stereo system that automatically generates texture-mapped 3D models. The key part of the system is a new method for tracking features and estimating the radiometric response function of the camera and the exposure difference between frames simultaneously. We model the global and nonlinear process that is responsible for the changes in image brightness rather than adapting to the changes locally and linearly which makes our tracking more robust to the change in brightness. The radiometric response function and the exposure difference between frames are also estimated in the process which enables our stereo process to deal with the varying brightness and generate radiometrically aligned textures.